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Abstract — We consider a network multicast example that 
relates the solvability of the multicast problem with the existence 
of an entropy function. As a result, we provide an alternative 
approach to the proving of the insufficiency of linear (and 
abelian) network codes and demonstrate the utility of non- 
Shannon inequalities to tighten outer bounds on network coding 
capacity regions. 

I. Introduction 

Traditionally, intermediate nodes in a communication net- 
work duplicate and forward packets towards their final desti- 
nation. Although such a store-and-forward scheme is simple 
to implement, it does not guarantee efficient utilization of 
available transmission bandwidth. The network coding ap- 
proach [1], [2] generalizes the traditional scheme by allowing 
intermediate nodes to forward mixed/encoded versions of all 
received data packets. Not only can this significantly increase 
network throughput in multicast scenarios, it can also provide 
robustness to Unk failure, and minimize cost of transmission. 

One fundamental problem in network coding is to un- 
derstand the capacity region and the classes of codes that 
achieve capacity. In the single session multicast scenario, the 
problem is well understood. In particular, the capacity region is 
characterized by max-flow/min-cut bounds and linear network 
codes are sufficient to achieve maximal throughput [2], [3]. 

However, things are much more complicated in more gen- 
eral multicast scenarios. It was recently proved that linear 
network codes are not sufficient for general multicast scenarios 
[3]. Furthermore, the capacity region achieved by network 
codes are also unknown. In fact, there are limited tools 
available to study the capacity region. 

One powerful tool is developed in [4], which outerbounds 
the capacity region by characterizing the intersection of a set of 
hyperplanes and the set of entropy functions. Unfortunately, 
it cannot be implemented in practice due to the lack of an 
explicit characterization of entropy functions for more than 
three random variables. One way to resolve this difficulty is 
by replacing the set of entropy functions with the set of non- 
negative submodular functions. The resulting bounds for the 
network coding capacity region can be quite loose. Recent 
work [5] based on matroid theory showed that application of 
the non-Shannon inequality [6], one can obtain a tighter bound 
for the capacity region (by obtaining a better outer bound for 
the set of entropy functions). 

In this paper, we construct a particular network and mul- 
ticast problem, which relates solvability to the existence of 
an entropy function on a set of four random variables. As 



a result, we provide an alternative approach to the proving 
of the insufficiency of linear (and abelian group) network 
codes, including time-sharing of such network codes. We also 
demonstrate the utility of non-Shannon inequalities to tighten 
outer bounds on network coding capacity regions. 

Section HI] provides (linear/group) network code fundamen- 
tals and relates the existence of such codes to the ranks of 
certain vector subspaces/group orders. Section [III] presents the 
main result, concerning a multicast problem induced by a 
function h with certain properties. Theorem [T] states that h is 
entropic if the induced multicast problem is solvable. Theorem 
|2] provides a converse, namely that the multicast problem is 
solvable if h is entropic. Section HVl explores the implications 
of this result, which include the insufficiency of linear or even 
(abeUan) group network codes, and the neccessity for non- 
Shannon inequalities. 

II. Networks, Codes and Capacity Regions 
A communication network will be modeled by a directed 
acyclic graph Q = {Af,£). The nodes u £ JV and directed 
edges e = (tail(e), hcad(e)) G £ respectively model the com- 
munication nodes and directed point-to-point communication 
Unks in the network. For e, / G £ we write / — > e as shorthand 
for hcad(/) = tail(e). Similarly, for / e u G Af, f ^ u 
means head(/) — u and u ^ f means tail(/) = u. 

For any network Q, a multicast requirement M = {S, O, D) 
is specified by: 

1) An index set S of all independent multicast sessions. 
Each session is a collection of data packets to be 
multicast to a prescribed set of destination nodes. 
A mapping O : S ^ J\f, where 0{s) is the originating 
node for multicast session s. 

A mapping D : S 2^ , where the set D{s) contains 
the destination nodes for the session s, all of which 
require the data of session s. 
Note: We do not specify any rate requirement. Given a mul- 
ticast requirement M, the objective is to efficiently multicast 
data for session s originating at 0{s) to all destinations D{s). 

For a given network Q and multicast requirement M, a 
network code solving the multicast problem is specified by 
a set of alphabets V = {V/,/GiSUiS} and a set of local 
coding functions $ = {0e: eGf}. 

Data transmission takes place as follows. The input gener- 
ated by session s is a symbol Vs G V^. The symbol transmitted 
along edge e G 5 is 

V,^(j),[Vf : f eS\j£J ^e), 



2) 



3) 



where to simplify notation, s ^ e indicates 0{s) e. We 
require that for every s e 5 and u E D{s), Vs can be uniquely 
determined from {Vf : f u} 

H{V,\Vf:feSU£J^u)^0. 

We will refer to a network code by $, with the alphabet V 
implicitly defined. Since the source, and hence edge symbols 
Vf are random variables, we can also refer to the code by the 
collection of random variables {Vf, / £ 5 U £}, where their 
joint distribution is implied by $. 

Definition 1. Given a network Q and a multicast requirement 
M, a rate-capacity tuple 

(Ajtj) = (As : s G 5, We : e G £) 

is admissible ;/ there exists a network code $ such that 

l0g|Ve| <CJe,Vee£ 

loglVsl > A,,Vse5 

Similarly, a tuple is asymptotically admissible ;/ there exists 
a sequence of network codes <I>*^"^ over alphabets V*-"-* and 
positive normalizing constants r{n) such that 



lim sup 

n — ^oo 

lim inf 



1 



n^oo r{n) 
1 



(n) 



log|Vi"'| <c^e,Vee£, 
log|Vi"'| > A„Vse5 



Furthermore, if the sequence of codes all belong to a par- 
ticular class (e.g. linear) we call the rate-capacity tuple 
asymptotically admissible by that class of codes. 

For a given network, multicast requirement and rate- 
capacity tuple, the multicast problem is to determine if the 
tuple is (asymptotically) admissible. Equivalently determine 
the set of all (asymptotically) admissible rate-capacity tuples. 

In the above formulation, there is no restriction on the 
choice of alphabets and local coding functions. However, 
network codes with neat algebraic structures may be preferred 
in practice, to reduce encoding and decoding complexity. 

Definition 2. A network code $ is linear over ;/ Vf is a 
vector space over F^, and the are linear 

Proposition 1. Suppose $ is linear Then there exists a vector 
space V with subspaces "V f, f £ £ [J S such that 



\S\ dim V — dimVs = dim V — dim 

5 ses 

is a subspace of Ve , Ve G 5 



ses 



n VfCV,,WseS,ueD{s) 

dimV - dimVs = log^ |Fs|,Vs G 5 
dimV-dimV/ < log |1//|,V/ G £ 



Proposition 2. Suppose that there exists a vector space V 
with subspaces ~V f, f G £ U S satisfying 

\S\ dim V - dim Vs = dim V - dim f] V, 

ses ses 

V/ is a subgroup ofVe,'^e G £ 

n V/ C Vs,Vs G G D{s) 

Then there exists a linear network code $ such that 
\ogq \Vf \ = dim V — dim V/ /or a// / G SU£. In other words, 
the rate-capacity tuple [Xg : s G 5, : e G 5) where Xg = 
(dim V — dim ) log q and ojg = (dim V — dim Vg ) log q is 
linearly admissible. 

The above two propositions show that linear network codes 
can be described by the ranks of a collection of vector sub- 
spaces. This idea can be extended to other algebraic structures. 

Proposition 3. Let G be a finite group with subgroups Gf,fE 
S U £. Suppose the following conditions are satisfied 



\G\/\f]Gs\^l[\G\/\Gs\ 

ses ses 

G f is a subgroup of Gc^e G £ 



Gf is a subgroup o/ G^, Vs G iS, w G D{s), 

Then there exists a network code such that the rate-capacity 
tuple (log|G|/|Gs| : s G 5, log |G|/|Ge| : e G £) is 
admissible. 

If the group G is also abelian, the group network code will 
be called abelian. 

III. Main Result 

Let h he a set function defined on the collection of all 
nonempty subsets of {1,2,3,4} such that 

Ml, 2, 3, 4) = Ml, 2, 3)^/^(1, 2, 4) = 

/i(l,3,4) = M2,3,4) = M3,4). (1) 

Then h induces a multicast problem MP(/i), together with 
an induced rate-capacity tuple, which we will now describe. 
There are three sessions Va,Vb,Vc all available at the same 
source node, with entropy rates H{Va) = ^(1). H{Vb) = 
/i(l, 2) - h{l) and H{Vc) = h{l, 2, 3) - h{l, 2). 

For convenience, we divide the underlying network into 
two parts. The first part, depicted in Figure [T] contains the 
source and generates a set of four network coded messages 
Vi,V2,V3,V4 which will be transmitted to the second half 
of the network, shown in Figure |2] To simplify notation, 
each edge is associated with a symbol denoting the network 
code message (and corresponding random variable) to be 
transmitted along that edge. If the link is capacitated, its 
capacity is given following the edge message symbol. 



Va,Vt,Vc 




V2 



Fig. 1. First part of the network 



The second part of the network is mainly composed of 
receivers, and is divided into several subnetworks. Figures 
|2(a)f|2(e)| each of which contains a receiver and a subset of 
Vi,V2,V3,V4 as input. Receivers are denoted by a solid square 
labeled by the set of source messages to be recovered. To 
simplify notation, all messages other than the four messages 
Vi,V2,V3,V4 will be denoted by a generic symbol W. As 
each receiver subnetwork has at most one such message, any 
confusion will be cleared from the context. 



V2 



Vi- 



(a) Subnetwork 1 

V3 Vs 



W,h{1.2) - h{2) 
(b) Subnetwork 2 



Va,Vt 




Va,Vi.,Ve 



ir, h(l,2,3)-/i(3) 
(c) Subnetwork 3 



W.h(i,j,k)-h(i.j) 
(d) Subnetwork 4 



V3 



V3 



ir,/i(i,3) - /i(3) 

(e) Subnetwork 5 
Fig. 2. Second part of the network 

The multicast problem that we are interested in is to 
determine if there is any network coding solution for the 
induced multicast problem. Equivalently, we wish to determine 
if the multicast problem MP(h) is (asymptotically) solvable 
or not. 

Theorem 1. Let h be a set function defined on the collection 
of all nonempty subsets of {1,2,3,4} satisfying ([T]). If the 
induced multicast problem MP(/i) is solvable, then h is the 
entropy vector for the random variables Vi,V2,V3,V4 (see 
Figure Q] and \2} of the network coding solution. Hence, h is 
entropic. 

Proof. Suppose {Va, Vb, Vc, Vi, ^2,^3, V4} is a solution to the 



multicast problem MP(/i). Then i?(Va) = h{l), H{Vb) = 
h{l, 2) - h{l) and H{Vc) = h{l, 2, 3) - h{l, 2). 

From the first half of the network, it is obvious that (Vi) < 
h{i) for i = 1, 2, 3, 4. Also, we can deduce 

H{V2,V:i) <H{V2,W,V:i) 
= H{V2,W) 
<H{V2)+H{W) 
<h{2) + h{2,3)~h{2). 

Similarly, H{V2,V4) < /i(2,4). 

To prove that h is the entropy vector for l^i, V2, V3, V4, we 
need to show that H{Vi,i e a) = h{a) for all a C {1, 2, 3, 4}. 
This is accomplished by Claims [T]-[9] below. □ 

Claim 1. H{Vi) = h{l) and H{Vi\Va) = 

Proof. From Figure [2(a)] H{Va\Vi) = in order to recover Va 
at the sink. Therefore, H{Vi) > H{Va) = h{l). As H{Vi) < 
h{l), the first part of the claim follows. Moreover, 

H{Vi\Va) = H{V,,Va) - H{Va) 

^H{V,)+H{Va\Vi)-H{Va) 

= H{V,)-H{Va) 

= 0. □ 

Claim 2. H{V2) = h{2) and H{Vi,V2) = h{l,2) 

Proof. From Figure |2(b)[ 

H{V2) + h{l, 2) - h{2) > H{V2) + H{W) 
> H{V2,W) 

>H{Va,Vb) 

= H{Va) + H{Vb) 

= h{l,2) 

As a result, H{V2) > h{2). Since H{V2) < h{2), we 
have H{V2) — h{2). On the other hand, from Figure |2(b)| 
H{Vi,V2) > H{Va,Vb) = 2). Finally, note that 

H{VuV2) < H{Vi,V2,Va,Vb,W) 

< H(V2,W,Va,Vb) + H(V,\Va) 
= H{V2,W)+ H{Va, Vb\V2,W) 

< H{V2)+H{W) 

= h{2) + h{l,2)-h{2) 

= h{l,2). □ 

Claim 3. HiVa) = h{3) and H{Vi) = /i(4) 
Proof. From Figure |2(c)[ 

HiVs) + h{l, 2, 3) - h{3) > HiVs) + H{W) 

>H{Va,Vb,V,) 

^h(l,2,3) 

Hence HiVs) > h{3) and since H{V3) < h{3), HiVs) = 
/i(3). Similai-ly, HiV^) = /i(4). □ 



Claim 4. HlVi, Vj) > h{i,j) for distinct i and j 
Proof. From Figure |2(d)| 

HiV,,V,) + Hi, J, k) - h{i,j) > HiV^, V,) + H{W) 

>H{Va,Vb,Vc) 

As a result, H(Vi,V.j) > h{i,j). □ 
Claim 5. H{Vi,V3) = h{l,3) and H(yi,Vi) = /i(l,4). 
Proof. From Figure |2(e)| 

HiVi,V3)<H{Vi,V3,W,Va) 

< H{V3,W) + H{Va\W,V3) + H{Vi\Va) 

< H{V3) + H(W) 
<h{3) + h{l,3)-h{3) 
-Ml, 3) 

Thus H{Vi,V3) = h{l,3). Similarly, H{Vi,Vi) = /i(l,4). 

□ 

Claim 6. H{V„ Vj,Vk) = h{i,j, k) 

Proof From Figure [2(d)l it is clear that H{Vi,Vj,Vk) > 
H{Va, Vb, Vc). On the other hand, as Vi,Vj, Vk are functions 
ofVa,Vb,Vc, therefore, H{V^,V,,Vk) < H{Va,Vb,V,) and 
hence 1^^, 14) = 14, V,) = j, fc). □ 

Claim 7. i/(V3,T4) = /i(3,4). 

Proof From Claim H we have H{V3,Vi) > /i(3,4). Since 
/i(3,4) = H{Va,Vb,Vc) and i/(14, ■t^4|K, Vt, 14) = 0, the 
claim then follows. □ 

Claim 8. H(14, 14, V3, 14) = 14, 14) = h{l, 2, 3, 4) 

Proof. Obvious □ 

Claim 9. i/(1^2,14) = h{2,3) and i/(14,14) = /i(2,4) 

Proo/ From Claim g] ff(14,14) > h{2,5) and i?(V2,14) > 
h{2,4). As i/(F2,14) < /i(2,3) and 71(^2,14) < /i(2,4) are 
proved in the beginning, the result then follows. □ 

Corollary 1. // the induced multicast problem MP(/i) is 
asymptotically solvable, then h ^ T\ which is the minimal 
closure of the set of all entropy functions for four random 
variables. 

Theorem [T] shows that if the induced multicast problem 
MP(ft,) is solvable, then h must be entropic. In the following 
theorem, we prove the converse. 

Theorem 2. Let h be entropic and that all the terms 
/i(l,2,3,4), /i(l,2,3), /i(l,2,4), /i(l,3,4), /i(2,3,4), /i(3,4) 
are equal to each other Then the rate-capacity tuple in the 
induced multicast problem MP{h) is asymptotically admissi- 
ble. 

Proof. As h is entropic, then there exists a set of random 
variables (t/i, C/2, U3, U4) whose entropy function is h. Hence, 



one can construct a sequence of groups G'^, with subgroups 
G^ji G {1,2,3,4} [7], [8] and normahzing constants r(fc) 
such that 

lim -^rrh'^ia) = h(a),aC {1,2,3,4} 
fc— »oo r[k) 

h''{l,2,3,4) = /i'=(l,2,3) = h''{l,2,A) = /i'^(l,3,4) 
= /i''(2,3,4) = h''{3,4). 

where h''{a) =\og\G^\/\ D^ea G^\. 

For any fc, the function h'^ induces a multicast problem 
MP(/i'=). If MP{h'') is solvable, then the theorem follows 
from limfe^oo h^{a)/r{k) — h{a). 

It remains to show that MP(/i'^) is solvable. For each fc, 
is entropic [7], [8]. In fact, {G^, G'^ : i e {1, 2, 3, 4}} induces 
a set of quasi-uniform random variables 1/1,1/2,1/3, C/4 such 
that 

1) For any a C {1,2,3,4}, {t/*^ : z e a} is uniformly 
distributed over its support. 

2) For any a,/3 C {1,2,3,4}, the conditional probability 
distribution of |J7f : i G a} given a particular instance 
of {[/f : i G /?} = {uf : i G /?} (with positive proba- 
bility) is constant over its support. 

By the quasi-uniformity of 1/1,1/2,1/3,1/^, it is 
straightforward to show that there exists another set of 
quasi-uniform random variables [/^ , J7^'"' , J7^' such that 
U^,Ul^,Ul: ai-e independent and all the terms H{Ui\U^), 
H{U^\U^), H{U^,U^\U^,U^), H{UtU^\U^,U^), 
H{U^,Ut,U^\U^,Ui,Ul), H{U^,U^,Ul\U^,Ut,U^) are 
equal to zero. 

It can be checked easily that U^,U^, are uniform and 
that iJ(C/^) = h^{l), H{U^) ^ h''{l,2) - h''{l) and 
H{U^) = /i'=(l,2,3) - h''{l,2). To show that MF{h'') 
is solvable, it remains to show that the auxiliary random 
variables (i.e., the generic message random variables W) can 
be constructed to satisfy the capacity constraints. 

This can be achieved by "data compression" of the in- 
puts subject to common side-information at both encoder 
and decoder. Since ,1/2 ,1/3 ,U^ are quasi-uniform, it is 
straightforward to verify that the "rate of compression" meets 
the (conditional) entropy lower bounds by using block coding. 
The result then follows. □ 



IV. Implications 

Theorem 3. There is a network and a multicast requirement 
such that the use of abelian network codes is suboptimal 
(including linear network codes, R-module codes, and time- 
sharing of such). 

Proof. First, we describe a set of four quasi-uniform random 
variables 1/1,1/2,1/3, U4 constructed using the projective plane 
described in [6]. The joint entropies of subsets of these random 



variables are as follows: 



V. Conclusion 



h{l) = h{2) = h{3) = (4) = log 13 
h{l,2) = log6 + logl3;/(,(3,4) = log 13 + log 12 
h{l,3) = h{lA) = h{2,3) = h{2,A) = logl3 + log4 
h{i, j, k) = log 13 + log 12 = /i(l, 2, 3, 4), V distinct i, j, k. 

Then it is clear that /i(l,2,3,4) = h{l,2,3) ^ /i(l,2,4) = 
/i(l,3,4) = /i(2,3,4) = h{3,4). Since /i is entropic, MP{h) 
is asymptotically solvable. In fact, it can be shown that h 
is an entropy vector of four quasi-uniform random variables. 
Therefore, using the same argument as used in the proof of 
Theorem |2] we can show that MP(ft,) is solvable. 

Suppose to the contrary that there is an abelian network 
code that solves the multicast problem. Then there exists an 
abelian group G with subgroups Gi,G2, 0^,04 such that 
h{a) = log \G\/\ fligQ, Gi\. By [8], h must satisfy the Ingleton 
inequality 

h{l, 2) + h{l, 3) + h{l, 4) + h{2, 3) + h{2, 4) > 

h{l) + h{2) + h{3, 4) + h{l, 2, 3) + h{l, 2, 4) (2) 

However, we can directly verify that h does not satisfy the 
Ingleton inequality. □ 

Corollary 2. There is a network and a multicast requirement 
for which abelian codes are (asymptotically) suboptimal. 

The multicast network constructed can not only be used to 
show the suboptimality of abelian network codes, it also shows 
that non-Shannon type information inequalities can be useful 
in determining the capacity region of network codes. 

Suppose /i(l,2,3,4) = /i(l,2,3) = /i(l,2,4) = 
/i(l,3,4) = /i(2,3,4) = /i(3,4). By Theorems [T] and E] we 
proved that the multicast problem MP (ft,) is asymptotically 
solvable if and only if ft G r|. Therefore, we can obtain an 
outer bound for network code capacity region by finding an 
outer bound for One simple outer bound is r4 defined as 
the set of all non-negative submodular functions. 

It was shown in [6] that r4 is a proper superset of r|. 
Therefore, by Theorem |2l the outer bound for the capacity 
region obtained in this manner is not tight. To tighten the 
bound for the capacity region, one approach is to tighten the 
bound for r4 by applying new non-Shannon inequalities [6] 
and in [9]. 

Consider the function ft defined as follows [6]: 

ft(l) = ft(2) = ft(3) = (4) = 2a > 
ft(l,2) = 3a;ft(3,4) = 4a 
ft(l, 3) = ft(l, 4) = ft(2, 3) = ft(2, 4) = 3a 
ft(i,j, fc) = 4a = ft(l,2,3,4), V distinct fc. 

Clearly, ft(l,2,3,4) = ft(l,2,3) = ft(l,2,4) = ft(l,3,4) = 
ft(2,3,4) = ft(3,4). It can be verified directly that ft G 
However, the non-Shannon information inequality obtained 
in [6] shows that h ^ V\. Therefore, the multicast problem 
MP (ft) is not asymptotically solvable. 



We have shown how to construct a multicast problem from 
a function ft that is solvable if and only if ft is entropic. This 
provides a useful link between entropy vectors and the capacity 
region for network codes. Using this approach, we provide 
an alternative proof of the insufficiency of linear (and abelian 
group) network codes, including time-sharing of such network 
codes. We also demonstrated the utility of the Zhang-Yeung 
inequality to tighten outer bounds on network coding capacity 
regions. 
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